Add Hyrise#883
Conversation
Closes #751 Hyrise is a research in-memory column-oriented database from HPI (https://github.com/hyrise/hyrise). It implements the PostgreSQL wire protocol, so the benchmark connects via psql and uses Hyrise's COPY ... WITH (FORMAT CSV) to load the standard ClickBench CSV dataset. The system is built from source via Hyrise's install_dependencies.sh and cmake/ninja; install_dependencies.sh requires Ubuntu 25.04 or newer. Since Hyrise has no on-disk persistence, the data size is reported as the total estimated segment size from the meta_segments meta table. Hyrise has limited SQL coverage (no DATE/DATETIME types, no REGEXP_REPLACE, no DATE_TRUNC). Queries that use unsupported functions are kept verbatim and will be reported as null in the result file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the build into a multi-stage Dockerfile (ubuntu:25.04 + gcc-15) so the
benchmark works on any Ubuntu host without polluting it with Hyrise's
toolchain. The runtime image only carries hyriseServer, libhyrise_impl.so,
libjemalloc.so, and a small set of shared-library deps (~250 MB).
Build args:
- HYRISE_REF (default master) — pin a Hyrise revision
- NO_LTO (default FALSE) — toggle LTO for faster development builds
Loading: drop create.sql and use hits.csv.json next to the data file as the
schema source. CREATE TABLE followed by COPY trips a Hyrise assertion
("set_immutable() should not be called on an empty chunk", chunk.cpp:125)
because COPY tries to seal the empty chunk left by CREATE TABLE; letting
COPY auto-create the table from the CSV meta avoids the issue.
run.sh: detect failed queries via psql's exit code rather than grepping the
output, so errors like "Invalid input error: Could not resolve function
'LENGTH'" are recorded as null. Hyrise lacks LENGTH, REGEXP_REPLACE,
DATE_TRUNC, and OFFSET, so 7 queries (Q28, Q29, Q39-Q43) are reported as
null; the remaining 36 succeed.
Tested locally on arm64: docker build produced a working image, hyriseServer
accepts psql connections, COPY loads a 1000-row sample, and run.sh produces
the expected 43 lines of [t1,t2,t3] output with nulls in the right slots.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Split the old monolithic benchmark.sh + run.sh into the new layout: install / start / stop / check / load / query / data-size, with benchmark.sh now a thin shim that exec's lib/benchmark-common.sh. BENCH_DURABLE=no because Hyrise has no on-disk persistence — the driver re-runs ./load on every cold cycle and rolls that wall-clock into the cold-try timing. The load script stages hits.csv (delivered by download-hits-csv into cwd) under data/ next to hits.csv.json, which the container exposes read-only at /data; the move is idempotent so subsequent cold-cycle reloads reuse the staged file. query keys off psql's exit code with ON_ERROR_STOP=1, so the seven queries Hyrise can't run (Q28, Q29, Q39–Q43 — LENGTH, REGEXP_REPLACE, DATE_TRUNC, OFFSET) propagate non-zero to the driver and get recorded as null instead of stealing the Time line psql still prints for the \timing meta-command. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bring in lib/benchmark-common.sh and lib/download-hits-* — the hyrise/benchmark.sh shim added in f94c8af exec's ../lib/benchmark-common.sh, but the branch had never been rebased onto main where lib/ lives, so the shim died with "No such file or directory" on every cloud-init run.
|
The run of Logs:
|
The benchmark run on c6a.4xlarge died linking `libhyrise_impl.so`: GCC's `-flto=auto` spawns one `lto1` per core, and 16 of them exhaust the machine's 32 GB RAM + 16 GB swap, so they get killed (`g++: fatal error: Terminated signal terminated program lto1` — earlyoom's SIGTERM). Hyrise's own CMake warns that LTO link times with GCC are very long, and the runtime difference does not justify a build that cannot complete on the reference machine. `NO_LTO=FALSE` stays available as a build arg. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`libhyrise_impl.so` links `libboost_container.so.1.83.0` and `libboost_date_time.so.1.83.0` dynamically (visible in the CI link command), but the runtime stage only installed boost-system and boost-thread, which do not depend on them — `hyriseServer` would fail to start with a missing shared library once the build succeeds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…shot The monolithic `COPY hits FROM '/data/hits.csv'` cannot work at full scale: Hyrise's CSV import reads the whole file into a single std::string and materializes every column as unencoded ValueSegments before dictionary-encoding them — roughly 200 GB peak for the 75 GB hits.csv. `COPY ... FROM` into an existing table appends chunks, so ./load now splits the file into 5M-row pieces and imports them sequentially, capping the transient overhead at one piece on top of the growing encoded table. The driver (BENCH_DURABLE=no) re-runs ./load on all 43 cold cycles, and re-parsing the CSV each time would blow any time budget. The first load therefore exports the table to Hyrise's binary format, which preserves the encoded segments and streams chunk-by-chunk on import; every reload imports that snapshot — the closest Hyrise equivalent of a durable system re-reading its own on-disk format after a restart. The export must go through `hyriseConsole`'s `export` command: server-side SQL `COPY TO` wraps the table in an MVCC `Validate` operator and BinaryWriter materializes the resulting ReferenceSegments as unencoded value segments, so the snapshot would be raw-sized and would re-import unencoded. The console is a separate process with its own catalog, so it performs the piecewise CSV import itself and the server only ever imports the snapshot (regenerating full-table statistics on add_table). The runtime image now ships `hyriseConsole` (plus libreadline/ libncurses), and the /data bind mount is writable for the snapshot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Analyzed the failed run and pushed three fixes. Why the run failed: the Docker build died linking Two more failures were waiting behind the build fix, both found by reading Hyrise's sources and fixed in this push:
Machine-size caveat: even with the piecewise load, the peak is bounded below by the final dictionary-encoded in-memory table, which I estimate at 30–50 GB — most likely above what c6a.4xlarge (32 GB RAM + 16 GB swap) can hold. I'd suggest running this on a memory-rich machine via the manual workflow (e.g. c6a.metal); the c6a.4xlarge PR run will probably fail at load with an earlyoom kill, and if so the standard machine should get an 🤖 Generated with Claude Code |
The PR benchmark always ran on c6a.4xlarge, but some systems cannot work there — e.g. Hyrise (ClickHouse#883), an in-memory DBMS whose encoded hits table alone exceeds the machine's 32 GB. The manual workflow can target any machine, but its runs are invisible on the PR until the results sink picks them up, and every re-run needs a manual dispatch. A maintainer can now add one or more `machine:<ec2-type>` labels to a PR (e.g. `machine:c6a.metal`) to override the default machine, one run per label. Adding such a label relaunches the benchmark immediately; other labels don't trigger anything. Labels can only be set by users with triage access, and every launch stays gated by the `benchmark-approval` environment, so the cost control is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The run of Logs:
|
|
Results for Logs:
|
Summary
hyriseServer,hyriseConsole,libhyrise_impl.so,libjemalloc.so.2). LTO is disabled by default: GCC's-flto=autolink spawns onelto1per core and OOMs a 32 GB machine.COPY ... FROMappends to an existing table), because Hyrise's CSV import reads the whole file into onestd::stringand materializes the table unencoded before dictionary-encoding it (~200 GB peak for the monolithic 75 GB file). Column types come fromhits.csv.jsonnext to each data file.hyriseConsole(exportbypasses the MVCCValidateoperator, so the snapshot keeps the encoded segments; SQLCOPY TOwould materialize it unencoded). All cold-cycle reloads (BENCH_DURABLE=no) import that snapshot, which streams chunk-by-chunk and regenerates statistics.meta_segments.estimated_size_in_bytessince Hyrise has no on-disk persistence.LENGTH,REGEXP_REPLACE,DATE_TRUNC, orOFFSET../querykeys off psql's exit code so those queries (Q28, Q29, Q39–Q43) are recorded asnullinstead of stealing the timing line that psql still prints.Closes #751
Test plan
SELECT COUNT(*),MIN/MAX(EventDate),AVG(UserID),COUNT(DISTINCT UserID),EXTRACT(MINUTE FROM EventTime)all return expected resultsnull(Q28/29 — unsupported functions, Q39–Q43 —OFFSET/DATE_TRUNC)Load time:line,Data size:line, 43 lines of[t1,t2,t3],csv_parser.cpp,import.cpp,binary_writer.cpp/binary_parser.cpp,console.cpp,sql_translator.cpp)🤖 Generated with Claude Code